Virtual computing environments allow multiple virtual machines (VMs) to run on a single physical platform (also referred to herein as a “host”) and to share physical resources. Some virtual computing environments enable configuration of VMs such that the total amount of memory designated for use by the VMs is larger than the actual amount of memory available on the host. Referred to as memory over-commitment, this feature enables the host to support the simultaneous execution of more VMs. To achieve over-commitment, the virtual computing environment simply gives a VM less memory than what the guest operating system (OS) in the VM believes it has.
Memory over-commitment is traditionally enabled using a technique known as ballooning, which is described in U.S. Pat. No. 7,433,951, the entire contents of which are incorporated by reference herein. A balloon is a resource reservation application that runs as a guest application in the VM or as driver in the guest OS that requests guest physical memory from the guest OS. After the guest OS has allocated guest physical memory for use by the balloon application, the balloon application is able to ultimately communicate information regarding the allocated guest physical memory to a hypervisor that supports the VM, which is then able to repurpose the host's system memory (also referred to herein as “machine memory”) backing the guest physical memory allocated to the balloon application. That is, since the balloon application only reserves guest physical memory but does not actually use it, the hypervisor can, for example, repurpose machine memory that backs such allocated guest physical memory for use by another VM without fear that the balloon application would write to the guest physical memory (and therefore the backed machine memory).
Another technique for memory management that is useful under memory over-commitment situations is referred to as page sharing. In this technique, the virtual computing environment identifies and eliminates redundant copies of guest physical memory pages across VMs. The virtual infrastructure maps the identical guest physical pages to the same machine memory page and enables copy-on-write behavior with regards to that machine memory page. This technique enables sharing of memory between VMs in cases where VMs may be running instances of the same guest OS, applications, and libraries, and have other data in common.
Unfortunately, there are applications and runtime environments that do not work well with conventional memory over-commitment in virtual computing environments. Java Virtual Machine (JVM) is one of the most widely used runtime environments in this category. JVMs typically have their own memory management mechanisms. Allocated Java objects sit in a JVM heap until the JVM runs out of heap space, and in this event, garbage collection sweeps the heap and recycles dead objects, which are the objects unreachable from the program. A memory-managed JVM runtime can be a problematic candidate for memory over-commitment because freed memory made available by garbage collection is typically held exclusively for the use of the runtime and therefore cannot be used by other applications running in the operating system or virtualized infrastructure. In this environment, attempts to over-commit the memory may result in lack of memory to support the JVM heap, causing a significant performance hit.
Such issues with memory over-commitment may be further exacerbated as a result of memory usage by certain applications running within a JVM. Notably, production-level applications typically use some form of caching to increase responsiveness and performance. However, application-level caches may continue to take up space in heap memory even though the JVM and underlying VM may be experiencing memory pressure. Some caching mechanisms have used “soft referenced” objects to permit a garbage collection to reclaim space from the cache. However, in practice, some JVMs may be slow to remove soft referenced objects even though the JVM may be under memory pressure. Further, applications generally have no control over the timing and scope of a garbage collection operation. For example, applications lack control over which soft referenced objects in the cache may be removed (i.e., garbage collected) and are unable to make a distinction between “hot” cache entries and “cold” cache entries. Further, multiple garbage collection operations may be needed to fully collect a soft referenced cache. Each garbage collection may occur at an inopportune time, each time causing the JVM to pause, thereby significantly impacting performance of the JVM and applications running therein.